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Abstract Body 


Background / Context: 

Within developing world contexts, education initiatives exist in environments that both pose 
challenges and offer opportunities for innovations. In Kenya, school fees were abolished in 2003, 
and since then, gross primary enrollment rates have risen to above 100% (World Bank, 2011). 
This dramatic increase in enrollment over a short period put considerable strain on the 
government school system. In 1998, the national student-teacher ratio was 28 to 1. In 201 1, it 
was 47 to 1 (World Bank, 2011). In addition to handling large classes, Kenyan teachers often 
deal with space and materials shortages that impair their ability to teach effectively (Sifuna, 

2007; UNESCO, 2005). Haiti is a historically bilingual nation, with both Haitian Creole and 
French as official languages. However, although all Haitians speak Haitian Creole, some 
estimates place the percentage of Haitians who speak French around 10%.’ Because French is 
not spoken in many Haitian homes, however, French is necessarily taught as a second language, 
with students in the first year of elementary school exposed to French oral language 
development. In addition, decades of political turmoil and natural disasters have decimated the 
education system. Perhaps unsurprisingly, given the high student-teacher ratios, limited teacher 
training, and lack of sufficient text materials, reading outcomes for students attending Kenyan 
and Haitian primary schools are generally poor, across languages (Bulat; 2014; Piper, 2010). The 
results of a series of assessments conducted over recent years converge on a common finding: 
children in these countries are not meeting the Ministry of Education’s benchmarks and on 
average read far below grade level (Bulat, 2014; Mugo et al., 2011; National Assessment Centre, 
2010; Onsomu et al., 2005; Piper, 2010; Piper & Mugenda, 2012; Wasanga, Ogle, & Wambua, 
2010). For example, in Kenya, the 2011 national Uwezo study found that just 57% of third- 
graders could read basic sentences, and only 30% a second-grade-level story (Mugo et al., 2011), 
whereas in Haiti a 2014 assessment of Haitian Creole and French literacy found that Grade 2 
students read on average fewer than 30 words per minute and had very little comprehension of 
what was read (Bulat, 2014). 

In addition to a growing interest in improving these poor reading outcomes in these countries, 
there is international focus on improving literacy, including through bilateral donors such as 
USAID and DFID. However, there is limited knowledge about how reading skills are acquired in 
multilingual environments such as in Sub-Saharan Africa or the Caribbean. This paper presents 
the findings of a longitudinal tracer study evaluating student literacy outcomes in Kenya over 
two years, three data collection points, and across two languages. Random selection and 
assignment methods were utilized to assign clusters of schools to a treatment and a control 
condition. The treatment group implemented a targeted literacy and numeracy instructional 
program. It also presents the findings of a two-year randomized control trial in Haiti evaluating 
the efficacy of a literacy curriculum for Haitian Creole and French. Both interventions included 
student books, teachers’ guides, and ongoing teacher professional development and supervision. 


1 DeGraff, M. (2013). Many hands make the load lighter: Haitian Creole and technology-enhanced active learning 
toward quality education for all in Haiti. Proceedings of the Sixth Conference of Learning International Networks 
Consortium (LINC 2013). 


SREE Spring 2015 Conference Abstract Template 


1 



Purpose / Objective / Research Question / Focus of Study: 

If children do not learn how to read in the first few years of primary school, they will struggle to 
complete the cycle and are at greater risk of dropping out. It is therefore crucial to identify and 
test interventions that have the potential of making a large impact, can be implemented quickly, 
and are affordable to be taken to scale by the Kenyan government. This is the goal of the PRIMR 
Initiative in Kenya and the ToTAL program in Haiti — to test various options for improving 
learning outcomes and instruction in primary schools, using a randomized controlled design. The 
design is essential to this contribution, as many pilot programs in the sector do not test the 
impacts of quality-improvement methods at a medium scale and with enough rigor to identify a 
causal impact. In this paper, we focus on early-grade literacy outcomes — reading fluency and 
comprehension — and how these literacy outcomes transfer between English and Kiswahili and 
Haitian Creole and French over time, and whether that differs in treatment and control schools. 

Setting: 

The PRIMR study was conducted in 547 government and nonfonnal slum schools in peri-urban 
and rural areas of Nairobi, Kisumu, Nakuru, and Kiambu counties in Kenya. Peri-urban regions 
are on the outskirts of urban areas — near enough that residents can commute to towns and cities 
via local transport, but still possessing many rural characteristics, such as agriculture being the 
predominant economic activity (Mandere, Ness, & Anderberg, 2010). Low-cost, private 
nonfonnal schools are common alternatives to public government schools, particularly at the 
primary level. The nonfonnal schools participating in PRIMR generally are characterized by low 
tuition rates (less than US$10 per month), substandard infrastructure (predominantly tin roofs 
and unfinished floors and walls), high student and teacher turnover, and lack of trained principals 
and teachers. The ToTAL study was conducted in 300 governmental and private schools in the 
three USAID development corridors. Schools varied in urbanity, again most with substandard 
infrastructure. Forty-percent of teachers had not completed secondary school, and most had 
received no teacher training prior to this study. 

Population / Participants / Subjects: 

Kenya PRIMR. During the January 2012 baseline, a clustered random selection of pupils 
resulted in a longitudinal sample of 1100 Grade 1 students in 117 total treatment and control 
schools. This is from a larger sample of more than 4300 students in more than 230 schools. At 
the October 2012 midterm, 827 longitudinal students remained in the schools they attended at the 
baseline, and at the endline, 657 students were present on the data collection day. Each pupil was 
assessed in 6 or more English and 6 or more Kiswahili literacy tasks, which slightly differed at 
each assessment period, as well as 7 mathematics subtasks. To our knowledge, this is the largest 
longitudinal dataset investigating literacy outcomes at multiple points in time in Sub-Saharan 
Africa that utilized random selection and assignment. Haiti ToTAL. Over 4,000 Grade 1 and 2 
students were tested at four points in time over two years, with students sampled from 200 
treatment and 80 control schools in the first year and 300 treatment and 80 control schools in the 
second year. Each student was administered ten tasks in Haitian Creole and between five and ten 
tasks in French. In both studies, student background information was collected, as were 


SREE Spring 2015 Conference Abstract Template 


2 



characteristics of the schools and the teachers that supported these teachers during the data 
collection periods. 

Intervention / Program / Practice: 

The interventions discussed here centered on improving teacher practices related to literacy 
acquisition, moving teachers beyond using whole-class oral repetition as their primary 
pedagogical approach toward research-supported strategies to improve bilingual literacy 
acquisition. The literacy arm of the PRIMR program included 150 structured lessons in both 
Kiswahili and English. Teachers received modest instructional aids, including pocket charts and 
flashcards with letters on them, and students received low-cost student books that aligned with 
the scripted lessons. In PRIMR classrooms, the student-to-textbook ratio was 1 : 1 rather than the 
standard 3:1 mandated by the national textbook policy and found in PRIMR schools at the 
baseline (Piper & Mugenda, 2012). The ToTAL program also included 150 structured lessons in 
each of Haitian Creole and French, although the focus of the French curriculum was oral 
language development. In addition to curricular materials, teachers received ongoing trainings 
and were supported by regular pedagogical coaching. The programs focused on letters, 
phonological awareness, and decoding skills. They also exposed students to controlled-text 
stories relevant to their local context, as well as stories for teachers to read aloud, with a heavy 
emphasis on comprehension strategies. Together, the lessons and materials were intended to 
move children who had not attended preschool and had little exposure to the alphabet from basic 
letter knowledge to full fluency and comprehension within one school year. 

A significant amount of time and technical expertise in both projects was spent on teacher 
professional development. The trainings provided brief substantive overviews of reading topics, 
then allowed ample time for teachers to practice with the scripted lesson plans and activities. In 
Kenya, each participating teacher and head teacher received 10 days of training during the first 
year of implementation, and for literacy, approximately 7 of those days was spent per year. TAG 
tutors and instructional coaches, responsible for supporting teachers in clusters of schools, 
received 15 days of training to ensure that they would be capable of guiding teachers as they 
implemented the program. In Haiti, teachers received 10 days of literacy-focused training, 
supplemented by two 5 -day regional training sessions and regular local groups meetings. Given 
their critical role in shaping instructional improvement nationally in these countries, these 
trainers are seen as critical to the program’s ongoing success and scalability. 

Research Design: 

PRIMR and ToTAF are randomized control trials of instructional interventions. In Kenya, 
random assignment to treatment or control (delayed commencement of program) group was 
conducted at the zone level — groups of 15 geographically proximate schools. Schools were 
randomly selected from clusters and zones, and the sample of schools in the baseline included 
one half of the total number of schools in each zone. A comparable approach was used in Haiti, 
with qualified schools randomly assigned to treatment conditions within corridor for the 
intervention, and from each of these pools of schools a sample randomly selected for assessment. 
Enumerators selected the pupils using simple random sampling by having all of the students in 
each grade line up and then randomly selecting ten students each from grades 1 and 2, using a 
sampling interval derived from the student population. 
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Data Collection and Analysis: 

The baseline data collection for PRIMR was completed in January 2012, at the beginning of the 
school year, and the year one (midpoint) assessment was completed in October 2012, with the 
endline data collected in October 2013. A battery of literacy tasks were undertaken for both 
English and Kiswahili using a version of the Early Grade Reading Assessment, or EGRA (see 
Gove & Wetterberg, 2011), adapted for use in Kenya. The assessments were conducted by 
Kenyan field staff who had worked with PRIMR lead implementer RTI International since 2007 
on several studies using EGRA. These assessors received five days of training before 
assessments commenced for both the baseline and year one studies. Interrater reliability scores 
were above 95% in both languages at all three data collection rounds. Previous research using the 
non-longitudinal dataset from the PRIMR study shows statistically significant impacts on 
English and Kiswahili outcomes (Piper, Zuilkowski & Mugenda, 2014); for the poor (Piper, 
Jepkemei & Kibukho, 2014); and particularly in schools provided additional instructional 
support by TAG tutors (Piper & Zuilkowski, 2014). Impacts ranged from .3 to .5 standard 
deviations (Piper & Mugenda, 2014). Baseline data collection for Year 1 of ToTAL was 
completed in December 2012, with endline completed in June, 2013. Data collection for Year 2 
occurred in October 2013 and June 2014. A battery of literacy tasks were undertaken for both 
Haitian Creole and French using a version of the Early Grade Reading Assessment, or EGRA 
(see Gove & Wetterberg, 2011), adapted for use in Haiti. The assessments were conducted by 
Haitian field staff who had received five days of training before assessments commenced for 
both the baseline and year one studies. 

Findings / Results: 

Analysis shows that PRIMR had a positive impact on the literacy outcomes of interest, including 
but not limited to letter sound identification fluency, oral reading fluency, the percentage of 
pupils who read at the MOE’s benchmark, and reading comprehension, although not on all 
combinations of language, grade, and school type. The longitudinal analysis is able to detennine 
how the language skills interact over time and across languages, and suggest that in some skills, 
pupils are transferring literacy skills from English to Kiswahili. This is unexpected based on the 
Western literature, which generally suggests language transfer in the other direction. Analysis 
similarly shows that ToTAL has a positive impact on key foundational skills of initial sound 
identification and letter sound identification, although not on all combinations of language and 
grade. What is interesting are patterns that suggest that instruction in the mother tongue of 
Haitian Creole led to gains in French on skills that were not explicitly taught. 

Conclusions: 

The meaningful impacts of these projects on student achievement have implications for teacher 
professional development in Kenya and Haiti, particularly as these governments move toward 
national scale-ups of the approaches. This research has shown that teachers can be sensitive to 
in-service teacher professional development (ITPD) if that ITPD is closely linked to the books 
and lesson plans used in schools, and if teachers are observed and supported frequently (Bulat, 
2014; Piper & Zuilkowski, in progress). 
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